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Abstract 

When foraging for information, users face a tradeoff 
between the accuracy and value of the acquired in- 
formation and the time spent collecting it, a problem 
which also surfaces when seeking answers to a ques- 
tion posed to a large community. We empirically study 
how people behave when facing these conflicting ob- 
jectives using data from Yahoo Answers, a community 
driven question-and-answer site. We first study how 
users behave when trying to maximize the amount of ac- 
quired information while minimizing the waiting time. 
We find that users are willing to wait longer for an ad- 
ditional answer if they have received a small number of 
answers. We then assume that users make a sequence of 
decisions, deciding to wait for an additional answer as 
long as the quality of the current answer exceeds some 
threshold. The resulting probability distribution for the 
number of answers that a question gets is an inverse 
Gaussian, a fact that is validated by our data. 

1 Introduction 

When searching for an answer to a question, people 
face a well known tradeoff between the accuracy of the 
acquired information and the time spent collecting it. 
The fact that it usually takes longer to find a better an- 
swer to a given question creates a dilemma which is in- 
herent in information seeking. Stopping the search for 
information early provides swift answers which might 
not be completely correct, whereas continuing to search 
for longer usually provides accuracy and completeness 
while sacrificing the timeliness of the answer. Worse, 
if answers to a question arrive at random intervals, a 
user seeking specific information that will inform a de- 
cision to be made faces the problem of having to wait 
for an uncertain interval of time in exchange for an an- 
swer which may or may not improve on what is already 
known. 

Rather than study these issues in a laboratory set- 
ting, we decided to do so in the natural context of the 
Web. Specifically, we studied user behavior within 



Yahoo Answers, a community-driven question-and- 
answer site with more than 21 million unique users. At 
Yahoo Answers, users post questions seeking to harvest 
the collective intelligence of others in the system. Once 
a user submits a question, it gets posted on the site. 
Other users can then submit answers to the question, 
which are also posted on the site. When the author of 
a question is satisfied with the answers he has received, 
he closes the question and thus terminates his search for 
answers. He then uses information from the answers 
he received to build his own "aggregate answer" to his 
question, that will potentially help him take a related 
decision. 

There are two aspects that users value with respect to 
the aggregate answers they obtain: accuracy and speed, 
and thus they try to maximize the accuracy of their ag- 
gregate answers without waiting too long. The accuracy 
of the aggregate answer depends on the accuracy of all 
individual answers that the question received. 

Anyone posting a question faces the following trade- 
off at any given point in time. He can either build his 
aggregate answer at a given time or wait for additional 
answers to arrive. If he waits, he may achieve a higher 
accuracy in the future, but also incurs a cost for wait- 
ing. The user's intent is to build his aggregate answer at 
the optimal stopping time. We take two complementary 
approaches to the analysis of user behavior with respect 
to speed-accuracy tradeoffs. 

Our first approach studies the speed-accuracy trade- 
off by using the number of answers as a proxy for accu- 
racy. In particular, we assume that the user estimates the 
accuracy of his aggregate answer by the number of an- 
swers that his question gets. Thus, he faces the follow- 
ing tradeoff: he prefers more to less answers, but does 
not want to wait too long. We analyze Yahoo Answers 
data to identify and quantify this tradeoff. Our first find- 
ing is that users are willing to wait more to obtain one 
additional answer when they have only received a small 
number of answers; this implies decreasing marginal re- 
turns in the number of answers. Formally, this implies 
a concave utility function in the amount of information. 
We then estimate the utility function from the data. 
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Our second approach considers the qualities of the 
individual answers without explicitly computing the 
cost of waiting. We assume that users decide to wait 
as long as the value of the current answer exceeds some 
threshold. Under this model, the probability distribu- 
tion for the number of answers that a question gets is an 
inverse Gaussian, which is a Zipf-like distribution. We 
use the data to validate this conclusion. 

The rest of the paper is organized as follows. Sec- 
tion previews related work. Section[3]describes Yahoo 
Answers focussing on the rules that are important for 
our analysis. Section |4] empirically studies the speed- 
accuracy tradeoff by using the number of answers as a 
proxy for accuracy. Section we focuses on how users 
assess quality. Section|6]concludes. 

2 Related Work 

This paper uses Yahoo Answers data to study informa- 
tion seeking behavior with respect to stopping when 
people face speed-accuracy tradeoffs. In this section 
we review related work. 

A number of papers have studied information seek- 
ing behavior on the Web. For instance, Efthimiadis con- 
siders the search for health information |7|, and Egusa 
et al. study how people search when asked to complete 
certain tasks |0. Russell et al. present a search-task 
taxonomy [ 16 1. However, these papers do not explicitly 
consider behavior with respect to stopping and speed- 
accuracy tradeoffs. 

The behavior of people with respect to stopping prob- 
lems has been studied extensively in the context of the 
secretary problem (6J. In the classical secretary prob- 
lem, applicants are interviewed sequentially in a ran- 
dom order and the goal is to maximize the probability 
of choosing the best applicant. The applicants can be 
ranked from best to worst with no ties. After each inter- 
view, the applicant is either accepted or rejected. If the 
decision maker knows the total number of applicants n, 
for large n the optimal policy is to interview and reject 
the first n/e applicants (where e is the base of the natu- 
ral logarithm) and then to accept the next who is better 
than these interviewed candidates ll6l . 

Experimental studies of the classical secretary prob- 
lem and variants show that people tend to stop too early 
and give insufficient consideration to the yet-to-be-seen 
applicants (e.g., [3 1). On the other hand, when there are 
search costs and recall (backward solicitation) of pre- 
viously inspected alternatives is allowed, people tend 
to search longer than the optimum lEOl . We note a 
key difference with the setting of information seeking: 
while in the secretary problem only one secretary can be 



hired, an information seeker can combine information 
from multiple sources to build a more accurate answer 
for his question. Moreover, in the secretary problem the 
decision maker does not face a speed-accuracy tradeoff, 
because time does not affect his payoff. 

The speed-accuracy tradeoff has been considered in 
various settings. One example is a setting where a 
group cooperates to solve a problem ifTTl . In psychol- 
ogy, on the other hand, the speed-accuracy tradeoff is 
used to describe the tradeoff between how fast a task 
can be performed and how many mistakes are made in 
performing the task (e.g., |5 19 1). 

There has been a number of empirical studies that 
use data from Yahoo Answers and other question- 
answering communities. Data from Yahoo Answers 
have been used to predict whether a particular answer 
will be chosen as the best answer [T], and whether 
a user will be satisfied with the answers to his ques- 
tion fi*4l . Content analysis has been used to study the 
criteria with which users select the best answers to their 
questions [ 13 1. Shah et al. study the effect of user par- 
ticipation on the success of a social Q&A site IfTTl . Aji 
and Agichtein analyze the factors that influence how the 
Yahoo Answers community responds to a question • 
Finally, various characteristics of user behavior in terms 
of asking and answering questions have been consid- 
ered in [9|. To the best of our knowledge, there have 
been no studies that consider user behavior in terms 
of the speed-accuracy tradeoff in question-answering 
communities. 

3 Yahoo Answers 

In this section we describe the rules of Yahoo An- 
swers that are important for our analysis, and explain 
why users often face speed-accuracy tradeoffs. We then 
briefly discuss the data we use. 

3.1 Rules 

Yahoo Answers is a question-and-answer site that al- 
lows users to both submit questions to be answered and 
answer questions asked by other users. When a user 
submits a question, the question is posted on the Ya- 
hoo Answers site. Other users can then see the question 
and submit answers, which are also posted on the site. 
According to standard Yahoo Answers terminology, the 
user that asks the question is called the asker, and a user 
answering is called an answerer. In this paper, we study 
the behavior of the asker, and thus the word user is used 
to describe the asker. 

Once the user starts receiving answers to his ques- 
tion, he can choose the best answer at any point in time. 
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After the best answer to a question is selected, the ques- 
tion does not receive any additional answers. We thus 
say that a user closes the question when he chooses the 
best answer. Closing the question is equivalent to ter- 
minating the search for answers to the question. 

Questions have a 4-day open period. If a question 
does not receive any answers within the 4-day open 
period, it expires and is deleted. However, before the 
question expires, the asker has the option to extend the 
time period a question is open by four more days. The 
time can only be extended once. However, most ques- 
tions are not extended, and in our analysis we only con- 
sider questions that were closed within the 4-day pe- 
riod. If the asker does not choose a best answer to his 
question within the 4-day open period, then the ques- 
tion is up for voting, that is, other Yahoo Answers users 
can vote to determine the best answer to the question. 

3.2 Askers Face Speed-Accuracy 
Tradeoffs 

We expect that the user is satisfied with the answers he 
received when he closes the question. The user then 
uses information from these answers to build his own 
aggregate answer to his question. Throughout the pa- 
per, we use the term aggregate answer to refer to the 
conclusion that the question author draws by reading 
the answers to his question. The aggregate answer is 
not posted on the Yahoo Answers site, and is often not 
recorded. 

Askers at Yahoo Answers are often asking questions 
to get information that will help them make a decision. 
In particular, the decision will be aided by the aggre- 
gate answer. In many cases, the asker prefers to decide 
sooner than later. For instance, this is usually the case 
when the asker is seeking information on which product 
to buy, because the sooner he decides what he is buy- 
ing, the sooner he will get the product, and the sooner 
he will derive value by using it. 

As an example, we can consider the following ques- 
tion from Yahoo Answers: "What is the best graph- 
ics card for gamers?" In this case, the asker specified 
his price range and what he wanted to use the card for. 
The answerers then wrote their opinions on which card 
would be the best for the asker. Then, the asker chose 
the best answer and stated his decision on which card 
he was going to buy. It is reasonable to expect that in 
this case the asker would derive strictly higher utility by 
having the graphics card sooner. 

This motivates the speed-accuracy tradeoff. In partic- 
ular, if the question has n answers, the asker can either 
make a decision now using these n answers, or make 
a decision later using more information. The cost of 



waiting is the cost of delaying the decision. 

We do not suggest that an asker closes his question 
because of the speed-accuracy tradeoff. We just expect 
that if an asker did close his question, then the closing 
time provides a good approximation to the time that the 
asker made his decision. We use the closing time in our 
analysis to understand how users behave with respect to 
speed-accuracy tradeoffs. 

We next briefly discuss why an asker may close his 
question. There are two motivations. First, this is a way 
of thanking the answerer for his time and effort. Sec- 
ond, an asker gets some points in the Yahoo Answers 
reputation system if he closes his question. Clearly, not 
everyone is sufficiently motivated to close his answers 
in this way, and as a result many askers do not close 
their questions. However, given that an asker did close 
his question, we expect that the closing time gives in- 
formation about how the asker behaved with respect to 
the speed-accuracy tradeoff. 

3.3 Data 

We use a Yahoo Answers dataset that was crawled in 
October 2008 by Aji and Agichtein Q- For each ques- 
tion in this dataset we know the time the question was 
posted, the arrival time of each answer to the question, 
and the time that the asker closed the question by se- 
lecting the best answer. 

For the purposes of this paper, we only consider 
questions for which the best answer was selected by the 
asker. The reason is that we are interested in the time 
that the asker terminates his search for information by 
closing the question. If the asker selects the best an- 
swer, this is the time that the best answer was selected. 
On the other hand, if the asker does not select a best an- 
swer, we have no relevant information (we do not know 
when and whether the asker built his aggregate answer). 

Furthermore, we restrict attention to questions that 
were open for less than 100 hours. This is motivated by 
the fact that questions are initially open for 4 days (96 
hours) and that most askers close their questions within 
this 4-day period. 

We thus use a subset of the originally collected data 
that consists of questions that were closed by the asker 
in less than 100 hours. This subset consists of 1,536 
questions. 

One could argue that there is no reason for a user 
to close his question before the 4-day open period is 
over. In particular, he could use the information from 
the answers he has received up to now, but close the 
question at the end of the fourth day. However, once a 
user takes a decision (e.g., buys a product), there is little 
or no value in getting additional information. Thus, if 
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Figure 1 : Histogram of the number of hours that ques- 
tions were open (before the asker chose the best an- 
swer). 

the user is planning to select a best answer to his ques- 
tion (in order to thank the answerer), he has no reason 
to wait until the 4-day period is over. Indeed, Figure Q] 
shows that most questions close a significant amount of 
time before the 4-day deadline. For instance, 29% of 
the questions close within just one day after the ques- 
tion was posted. 

4 Speed-Quantity Tradeoff 

A user wants to get an accurate aggregate answer (that 
will potentially help him take a decision) without wait- 
ing too long. The accuracy of the user's aggregate an- 
swer is subjective and hard to measure. In this sec- 
tion we use the number of answers as an approximation 
of accuracy. Thus, we expect that a user's utility in- 
creases in the total number of answers that his question 
receives, and decreases in the time he waits for answers 
to arrive. 

Section 14.11 develops our hypotheses drawing on a 
utility model. In Section 1431 we test our first hypothe- 
sis. In Section l4~3l we introduce a discrete choice model, 
which we estimate in Section l4~4l to test the remaining 
hypothesis. Finally, in Section |431 we discuss the form 
of the utility function. 

4.1 Utility Model 

Let n be the total number of answers at the time the 
user builds his aggregate answer. We assume that the 
user gets utility u(n). Furthermore, we assume that the 
user incurs a cost e(t) for waiting for time t. Thus, the 



If u(n + 1) — u(n) < E[c(T)], then close the question 
If u(n + 1) - u(n) > E[c(T)], then wait 



Figure 2: Myopic decision rule. 

user is seeking to maximize u(n) — c(t). 

4.1.1 Myopic Decision Rule 

Suppose that n answers have arrived. The user can ei- 
ther terminate his search by choosing the best answer, 
or wait for additional answers. If he terminates his 
search now, he can build his aggregate answer using 
the n answers that he has received, and thus get utility 
u{ri). If he chooses to wait and a new answer arrives 
t time units later, then he will have n + 1 answers, but 
will have also incurred a cost c(t) for waiting. His util- 
ity will then be u(n + 1) — c(t). The user is better off 
stopping if u(n + 1) — u(n) < c(t), and continuing if 
u(n + 1) — u(n) > c{t). In words, the user decides 
to close the question if the cost of waiting for one more 
answer exceeds the incremental benefit of having one 
more answer. 

Our previous description assumes that the user knows 
when the next answer will arrive, which is not the case 
in reality. More generally, let T be a random variable 
that describes the user's belief on how long it will take 
until the next answer arrives. Then, the user is better off 
closing the question if u(n + 1) — u(n) < E[c(T)], and 
continuing if u(n + 1) — u{n) > E[c(T)]. 

The strategy we just described is myopic, since it as- 
sumes that a user decides whether to wait (i.e., not to 
close the question) by only considering whether he is 
better off waiting for one more answer. Alternatively, 
if the user knew when each answer is going to arrive 
in the future, we could consider a global optimization 
problem: if the i-th answer is expected to arrive at time 
tj, the user would choose to close the question at the 
time tj that maximizes u(j) — c(tj). However, in the 
context of Yahoo Answers it is impossible for users to 
know when all future answers will arrive. It is thus 
more realistic to assume that users myopically optimize 
as randomness is realized. 

We summarize the myopic decision rule in Figure |2] 
It implies that a user is more likely to close the question 
when u(n + 1) — u(n) is small and/or E[c(T)] is large. 

4.1.2 Hypotheses 

We next develop our hypotheses building on the my- 
opic decision rule. Our hypotheses can be grouped in 
two categories. The first category is based on the as- 
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sumption that the marginal benefit of having one more 
answer decreases as more answers arrive; the second 
considers how users estimate when the next answer will 
arrive. 

The user's valuation for having n answers is u(n). 
We expect that u(n) is concave, i.e., the marginal bene- 
fit of having one more answer decreases as the number 
of answers increases. According to the myopic decision 
rule (Figure|2]l, the user is more likely to close his ques- 
tion when u(n + 1) — u(n) is small. Since we expect 
that u(n + l) — u(n) is decreasing in n, the user is more 
likely to close his question when n is large, i.e., when 
he has already received a large number of answers. We 
test this in two ways, outlined in Hypotheses[TJand[2] 

Hypothesis 1. The amount of time that a user waits 
since the arrival of the last answer before closing his 
question is decreasing in the number of answers that 
the question has received. 

Hypothesis 2. A user is more likely to close his ques- 
tion if the question has received many answers. 

The user believes that the time until the next answer 
arrives is described by some random variable (which 
can be degenerate if he is only using an estimate). It 
is reasonable to assume that a user forms his belief us- 
ing the information available to him, that is, the arrival 
times of previous answers and the time he has waited 
since the last answer arrived. 

A particularly important summary statistic is the last 
inter-arrival time, i.e., the time between the arrivals of 
the two most recent answers. The last inter-arrival time 
is an estimate of the inverse current arrival rate of an- 
swers. Thus, the user may use the last inter-arrival time 
as an estimate of the next inter-arrival time, i.e., the time 
between the arrival of the last answer and the next an- 
swer. More generally, the user may form a belief on 
the next inter-arrival time that depends on the last inter- 
arrival time in some increasing fashion. Then, if the last 
inter-arrival time is large, the user expects to wait a long 
time until he receives another answer, thus incurring a 
large waiting cost. This encourages the user to close the 
question now. This is the context of Hypothesis [3] 

Hypothesis 3. A user is more likely to close his ques- 
tion if the last inter-arrival time is large. 

This hypothesis is based on the assumption that the 
last inter-arrival time may be used as an estimate for 
the next inter-arrival time. However, if a long time has 
elapsed since the last answer arrived (e.g., a longer pe- 
riod than the last inter-arrival time), the user becomes 
less certain about this estimate. The increased uncer- 
tainty may lead him to expect a greater waiting cost un- 
til the next answer arrives. In turn, this encourages the 



Corelation coefficient -0.148*** 
95% confidence interval [-0. 196, -0.098] 
Observations 1,536 



Table 1: Correlation between the number of answers 
(TotalAnswers) and the time that the user waits before 
closing the question (ElapsedTime). *, ** and *** de- 
note significance at 1%, 0.5% and 0.1% respectively. 

user to close the question, as is outlined in Hypothesis 

SI 

Hypothesis 4. A user is more likely to close his ques- 
tion if a long time has elapsed since the most recent 
answer arrived. 

Hypothesis [TJ is tested in Section l4~2l Then, in Sec- 
tion 14.31 we introduce a discrete choice model, which 
we estimate in Section l4~4l to test Hypotheses [2JE] and 

SI 

4.2 Time Between Last Arrival and 
Closure 

In this section, we test whether a user waits longer be- 
fore closing his question when the question has received 
a small number of answers (Hypothesis[T]i. 

For every question we consider the following vari- 
ables: 

• TotalAnswers: the total number of answers that the 
question received. This is the number of answers 
at the time that the asker closed the question. 

• ElapsedTime: the time between the arrival of the 
last answer and the time the user closed the ques- 
tion. 

We test for correlation between TotalAnswers and 
ElapsedTime. The results are presented in Table Q] We 
find that TotalAnswers and ElapsedTime are negatively 
correlated, bringing support for Hypothesis[T] 

Table Q] suggests that the user is willing to wait more 
(and incur more cost from waiting) for an additional 
answer if only a few answers have arrived up to now. 
This implies that the marginal benefit of having one ad- 
ditional answer decreases as the number of answers in- 
creases. 

4.3 Model Specification 

In this section, we introduce a logit model, which is 
estimated in Section l4~4l to test Hypotheses [2] [3] andSl 
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A user posts a question. Then, at various points in 
time he revisits Yahoo Answers to see the answers that 
his question has received, and decides whether to close 
the question by selecting the best answer. We are inter- 
ested in the probability that the user closes the question 
during a given visit. For every visit we consider the 
following variables: 

• p: the probability that the user closes the question 
during the visit. 

• n: the number of answers that the question has 
received by the time of the visit. 

• I: the last inter-arrival time, i.e., the time between 
the arrivals of the two most recent answers (at the 
time of the visit). 

• w: the time since the last answer arrived, i.e., the 
time that the user has been waiting for an answer 
since the last arrival. This is equal to the difference 
between the time of the visit and the arrival time of 
the most recent answer. 

Recall the utility model introduced in Section l4~Tl If 
the user closes the question at n answers, his utility is 
u{ri). Suppose that the user believes that the next an- 
swer will arrive in time T, where T is some random 
variable. Moreover, we assume that the user uses the 
last inter-arrival time I and the time since the last an- 
swer w to form his belief; that is T depends on I and w, 
and we write T(l,w). Then, the user expects to obtain 
utility u(n + 1) — E[c(T(Z, w))] from waiting. Accord- 
ing to the myopic decision rule (Figure [2]), the user de- 
cides whether to close the question or not depending on 
which of the expressions u(n), u(n+l)— E[c(T(Z, w))] 
is larger. 

We now perturb u(n) and u(n + 1) — E[c(T(Z, w))} 
with some noise. In particular, we assume that the 
user's utility is 

u(n) + e 
if he closes the question, and 

u(n + l) -E[c(T(Z,io))]+ei 

if he waits for the next answer. Thus, the probability of 
closing the question is 

p = P[u(n) + e > u(n + 1) - E[c(T(/, to))] + ej 
= P[ei - e < E[c(T(L w))] - {u{n + 1) - u(n))] 
= F(E[c(T(l, to))] - (u(n + 1) - u(n))), 

where F is the cumulative distribution function of c\ — 



Suppose that e an d £ i are independent type 1 ex- 
treme value distributed. Then, the difference ei — eo is 
logistically distributed, and 

p = A(E[c(T(l, to))] - (u(n + 1) - u(n))), 

where 



is the logistic function. 

The previous argument gives rise to the logit model, 
a standard discrete choice model in microecomics (see 
e.g., HD0 

In Section |4~4l we estimate the following model: 

P = A(a + f3 1 -n + f3 2 -l + (3 3 -w), (1) 

so that 

E[c(T(l,w))]-(u(n+l)-u(n)) = a+f3 r n+p 2 -l+p 3 -w. 

This implies that the marginal benefit of having one 
more answer when n answers have arrived is 

u(n + 1) — u(n) = a u — P\ ■ n (2) 

and the expected cost of waiting for the next answer is 

E[c(T(l,w))] = a c + /32-l + /3 3 -w (3) 

such that 

a c - ol u = a. 

Equations (fJI and d3) are used in Section l431 to interpret 
the estimated parameters of ([B. 

4.4 Model Estimation 

We now use logistic regression to estimate ((T), i.e., we 
estimate the probability that a user closes his question 
as a function of (i) the number of answers (n), (ii) the 
last inter-arrival time (I), and (iii) the time that the user 
has waited since the last answer arrived (w). We find 
that the probability of closing the question increases 
with all three variables, supporting Hypotheses|2][3] and 
^respectively. 

We estimate dTJ assuming that users visit Yahoo An- 
swers to check for new answers to their question every 
hour after the last answer arrived. The maximum like- 
lihood estimators are given in Table [2] All parameter 
estimates are statistically significant at the 0.001 level. 

'if we assume a different distribution for eo and ei, we get a dif- 
ferent discrete choice model. The logit model is widely used because 
of its simplicity. Another widely used model is the probit model, 
which assumes that eo an d e l 316 normally distributed. We note that 
the probit model gives the same qualitative results as the logit model 
for out dataset. 
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Estimate 
a -4.408*** (0.0603) 

fa 0.027*** (0.0005) 

fa 0.028*** (0.0002) 

fa 0.021*** (0.0001) 

Observations 54,914 



Table 2: The effect of the number of answers, the last 
inter-arrival time, and the time since the last answer on 
the probability of closing the question with p as the de- 
pendent variable. *, ** and *** denote significance at 
1%, 0.5% and 0.1% respectively. Standard errors are 
given in parenthesis. 

We also used a generalized additive model iflOl to fit the 
data, which suggested that the assumed linearity in (fl} 
is a good fit for the data. 

It is worth noting that our results do not heav- 
ily depend on our assumption that users check for 
new answers every hour. In particular, we get sim- 
ilar estimates for fa, fa and fa, if we assume that 
users check for answers every 2 hours, every 5 hours, 
or every 30 minutes. For instance, if we assume 
that users check for new answers every 2 hours, we 
get (fa, fa, fa) = (0.026,0.027,0.022) instead of 
(fa, fa, fa) = (0.027,0.028,0.021). 

We can now draw qualitative conclusions by consid- 
ering the signs of the estimated coefficients. We use the 
fact that the sign of a coefficient gives the sign of the 
corresponding marginal effect (since the logistic func- 
tion is increasing). 

First, the probability of closing the question is greater 
when more answers have arrived, which supports Hy- 
pothesis [2] This implies that the marginal benefit of 
having one additional answer decreases as the number 
of answers increases, and is consistent with Table [T] of 
Section l4~2l 

Second, the probability of closing the question is 
greater when the last inter-arrival time is greater, which 
supports Hypothesis [3] The inverse of the inter-arrival 
time gives the rate at which answers arrive. Thus, when 
the last inter-arrival time is large, i.e., there is a large 
time gap between the last answer and the answer before 
it, the user may expect that he will have to wait a long 
time until he receives the next answer. This perceived 
high cost of waiting may encourage the user to close the 
question sooner when the last inter-arrival time is large. 

Third, the probability of closing the question is 
greater when more time has elapsed since the last an- 
swer, which supports Hypothesis |4] As more time 
elapses since the last answer, the uncertainty increases, 
since the user does not know when the next answer will 



arrive. The increased uncertainty may lead him to ex- 
pect a greater waiting cost until the next answer arrives. 
In turn, this encourages the user to close the question. 

4.5 Utility and Cost 

The following lemma establishes a specific quadratic 
form for the utility function. 

Lemma 1. Tjf © holds, then 

«(n)= (a a + (j)n-^n 2 +u(0). (4) 
Proof. If © holds, then 

n— 1 

u(n) = u(0) + ^(a u - fai) 
i=0 

= \at u + — I n- —n + u(Q) 

□ 

We observe that the utility function given in (0]i is 
concave on [0, oo) for any value of a u as long as fa > 
0, which is the case for our dataset since fa = 0.027 
(see Table |2). Moreover, for any fixed fa > and 
a u > 0, the utility is unimodal: it is initially increas- 
ing (for n < [a u /fa + 0.5J) and then decreasing (for 
n > \a u /fa + 0.5] ). The latter may occur due to infor- 
mation overload; after a very large number of answers, 
the benefit of having one more answer may be so small 
that the cost of reading it exceeds the benefit, thus cre- 
ating a disutility to the user.Nevertheless, since ques- 
tions at Yahoo Answers rarely get a very large number 
of answers, the utility function given by may be in- 
creasing throughout the domain of interest if a u /fa is 
sufficiently large. 

We note that from (fl} we estimate fa and a, but can- 
not estimate a u . For the sake of illustration, we plot the 
estimated utility u(n) for various values of a u in Fig- 
ure [3] assuming that u(0) = 0. A reasonable domain 
to consider is [0, 50], since questions rarely get more 
than 50 answers. We observe that when a u is small 
(e.g., a u = 1), then the estimated u(n) is decreasing 
for large values of n within the [0, 50] region; suggest- 
ing an information overload effect. On the other hand, 
for a u & {2, 3, 4}, the estimated utility function is in- 
creasing throughout [0, 50]. Moreover, as a u increases, 
the curvature of the estimated utility decreases, some- 
thing we can also conclude from 

We next consider the cost of waiting. Equation (01 
suggests that the expected cost increases linearly in both 
the last inter-arrival time I and the time since the last 
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Figure 3: Estimated u(n) for a u 6 {1, 2, 3, 4}. 

answer arrival w. However, it is not possible to get a 
specific form for the function c(t) in the way we did for 
u(n) in LemmaQ] because we do not have any informa- 
tion on T(l, w). If we make assumptions on T(l, w), we 
can draw conclusions about c(t). For instance, if we as- 
sume that the user is using a single estimate r(l, w) on 
the time until the next answer arrives, then (01 implies 
that 



c(r(l, w)) 



(3 2 l + few. 



Moreover, if the estimate t(1, w) is linear in I and w we 
conclude that the cost of waiting is linear. On the other 
hand, a concave cost would be consistent with a convex 
estimate, and a convex cost would be consistent with a 
concave estimate. 



5 Assessing Quality 

Our previous analysis considers how the decision prob- 
lem of the user depends on the number of answers and 
time. There is a third aspect that affects the user's de- 
cision to close his question: the quality of the answers 
that have arrived up to now. In this section, we use an 
alternative model that incorporates quality, but does not 
incorporate time and the number of answers in the detail 
of Section|4] Our approach here is inspired by lfl5l[T2l . 

Let X n be the value of the n-th answer. This is in 
general subjective, and depends on the asker's interpre- 
tation and assessment. We assume that the value of an 
answer depends on both its quality and on the time that 
the user had to wait to get it. For instance, X n may be 
negative if the waiting time was very large and the an- 
swer was not good (according to the user's judgement). 
We model the values of the answers as a random walk, 



X n +i — X n + Z n , 



(5) 



where the random variables Z n are independent and 
identically distributed. For instance, if the user just got 
a high quality answer, he believes that the next answer 
will most likely also have high quality. Similarly, if the 
user did not have to wait long for an answer, he expects 
that the next answer will probably arrive soon. We note 
that <(5j is consistent with the availability heuristic 1 18 1, 
which leads individuals to judge the frequency of an 
event by how easily they can bring an instance to mind. 

Every time that a user sees an answer, he derives util- 
ity that is equal to the answer's value. We assume that 
the user discounts the value he receives from future an- 
swers according to a discount factor 8. Let V(x) be the 
maximum infinite horizon value when the value of the 
last answer is x. Then, 

V(x) =x + max{0, 6 ■ E(V(x + Z))}. 

In particular, the user decides to close the question if the 
value of closing exceeds the value of waiting for an ad- 
ditional answer. If he closes the question, the user gets 
no future answers, and thus gets future value equal to 0. 
On the other hand, if the user does not close the ques- 
tion, he gets value E(V(x + Z)) in the future, which 
he discounts by 6. Depending on which term is greater, 
the user decides whether to close the question or not. 

We observe that V(x) is increasing in x, which im- 
plies that E(V(x + Z)) is increasing in x. We conclude 
that there exists a threshold x* such that it is optimal 
for the user to stop (i.e., close the question) when the 
value of the last answer is smaller than x* and to con- 
tinue when the value of the last answer is greater than 
x*. The threshold x* satisfies E(V(x* + Z)) = 0. 

From an initial answer value, the user waits for addi- 
tional answers, with values following a random walk as 
specified by Q, until the value of an answer first hits 
the threshold value. Thus, the number of answers un- 
til the user terminates the search is a random variable. 
In the limit of true Brownian motion, the first passage 
times are distributed according to the inverse Gaussian 
distribution ||T5l . Then, the probability density of the 
number of answers to a question is given by 



/(*) 



—.v '- 

2tt 



exp 



A 



2fi 2 x 



(x - fi) : 



(6) 



where /i is the mean and A is a scale parameter. We note 
that the variance is equal to /i 3 / A. 

We use the dataset to test the validity of (|6). We find 
that the maximum likelihood inverse Gaussian has // = 
6.1 and A = 5.8. Figure|4]shows the empirical and fitted 
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Figure 4: Empirical and inverse Gaussian fitted cumu- 
lative distributions. The points are the empirical cumu- 
lative distribution function of the number of answers. 
The curve is the cumulative distribution function of the 
maximum likelihood inverse Gaussian. 

cumulative distribution functions. We observe that the 
inverse Gaussian distribution is a very good fit for the 
data. 

An important property of the inverse Gaussian distri- 
bution is that for large variance, the probability density 
is well-approximated by a straight line with slope -3/2 
for larger values of a; on a log-log plot; thus generating 
a Zipf-like distribution. This can be easily seen by tak- 
ing logarithms on both sides of ©. In Figure|5]we plot 
the frequency distribution of the number of answers on 
log-log scales. We observe that the slope at the tail is 
approximately -3/2. 

6 Conclusion 

This paper empirically studies how people behave when 
they face speed-accuracy tradeoffs. We have taken two 
complementary approaches. 

Our first approach is to study the speed-accuracy 
tradeoff by using the number of answers as a proxy 
for accuracy. In particular, we assume that the user ap- 
proximates the accuracy of his aggregate answer by the 
number of answers that his question gets. Thus, the 
user faces the following tradeoff: he prefers more to 
less answers, but does not want to wait too long. We an- 
alyze Yahoo Answers data to identify and quantify this 
tradeoff. We find that users are willing to wait longer 
to obtain one additional answer when they have only 
received a small number of answers; this implies de- 
creasing marginal returns in the number of answers, or 
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Figure 5: The frequency distribution of the number of 
answers on log-log scales. 

equivalently, a concave utility function. We then esti- 
mate the utility function from the data. 

Our second approach focuses on how users assess 
the qualities of the individual answers without explic- 
itly considering the cost of waiting. We assume that 
users make a sequence of decisions to wait for another 
answer, deciding to wait as long as the current answer 
exceeds some threshold in value. Under this model, the 
probability distribution for the number of answers that 
a question gets is an inverse Gaussian, which is a Zipf- 
like distribution. We use the data to validate this con- 
clusion. 

It remains an open question how to combine these 
two approaches in order to study the speed-accuracy 
tradeoff by jointly considering the number of answers, 
their qualities, and their arrival times. 

We conclude by noting that our results could be used 
by Yahoo Answers or other question-answering sites 
to prioritize the way questions are shown to potential 
answerers in order to maximize social surplus. The 
key observation is that a question receives answers at a 
higher rate when it is shown on the first page at Yahoo 
Answers. On the other hand, the rate at which answers 
are received also depends on the quality of the question. 
Using appropriate information about these rates as well 
as the utility function estimated in this paper, the site 
can position open questions with the objective of maxi- 
mizing the sum of users' utilities. 
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